Quality-Sensitive Test Set Selection for a Speech Translation System
نویسندگان
چکیده
We propose a test set selection method to sensitively evaluate the performance of a speech translation system. The proposed method chooses the most sensitive test sentences by removing insensitive sentences iteratively. Experiments are conducted on the ATR-MATRIX speech translation system, developed at ATR Interpreting Telecommunications Research Laboratories. The results show the effectiveness of the proposed method. According to the results, the proposed method can reduce the test set size to less than 40% of the original size while improving evaluation reliability.
منابع مشابه
High-quality Speech Translation for Language Learning
In this paper, we describe a translation framework aimed at achieving high-quality speech translation within restricted conversational domains. Towards this goal, we developed an interlingua-based approach, in which a generation-based method is augmented with an examplebased method to improve system robustness, even with imperfect inputs due to speech recognition errors. The framework is integr...
متن کاملAnnotating data selection for improving machine translation
In order to efficiently improve machine translation systems, we propose a method which selects data to be annotated (manually translated) from speech-to-speech translation field data. For the selection experiments, we used data from field experiments conducted during the 2009 fiscal year in five areas of Japan. For the selection experiments, we used data sets from two areas: one data set giving...
متن کاملSegmentation and punctuation prediction in speech language translation using a monolingual translation system
In spoken language translation (SLT), finding proper segmentation and reconstructing punctuation marks are not only significant but also challenging tasks. In this paper we present our recent work on speech translation quality analysis for German-English by improving sentence segmentation and punctuation. From oracle experiments, we show an upper bound of translation quality if we had human-gen...
متن کاملThe Effect of Private Speech and Self-Regulation on Translation Quality among Iranian Translation Students: A Mixed-Methods Study
The current study presents findings from a mixed-methods study of investigating the self-regulatory role of private speech (self-talk) on students’ translation quality. The aim of the study was to validate the adapted version of a self-verbalization questionnaire. The construct validity and reliability of the scale were supported by the CFA which revealed that all items reached the acceptable f...
متن کاملPhd Defense Presentation 2219 Engineering Building " Da a Analy I and Selec Ion for S a I Ical Macine Tran La Ion "
Statistical Machine Translation has received significant attention from the academic community over the past decade. This research has led to significant improvements in machine translation quality. As a result, it is widely adopted in the industry (Google, Microsoft, Twitter, Facebook, ...etc.) as well as the government (http:/ /nist.gov). The biggest factor in this improvement has been the av...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2002